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(54) Image pick-up device, image display device and information recording medium comprising 
afisheyelens 



(57) An image is picked up by a camera (2) compris- 
ing a fisheye lens (1) having a relationship of 
h = n • f • tan(0/m) , wherein h is the height of an image 
of a subject at a certain point, f is the focal distance of 
the fisheye lens. 0 if a field angle, m has a value of 1 .6< 
m ^3, and n has a value of m-0.4 ^n <, m + 0.4 , and the 



image data of which is output from the camera (2), is 
converted into a plane image by an image data process- 
ing unit (3), and this converted image is then output to a 
monitor unit (4). Preferably, n and m both equal 2. 
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Description 

[0001] The present invention relates to an image 
pick-up device comprising a f isheye lens, an image dis- 
play device and an information recording medium, all of 
which can obtain a high-quality converted image when 
an image picked up by the fisheye lens is converted into 
a plane image. - 

[0002] A monitoring system using a camera which 
enables product examination at a plant or construction 
work at a construction site to be monitored from a 
remote place has recently been developed. In this mon- 
itoring system, depending on what is monitored, capa- 
bility of monitoring a wide range at a limited number of 
cameras is desired. To realize this, the development of 
a monitoring system comprising a fisheye lens which 
can pick up an image of all the directions of the field of 
view around the optical axis at a field angle of at least 
90° in each direction with respect to the optical axis is 
under way. 

[0003] Use of this fisheye lens makes it possible to 
obtain an image of all the space with a single camera. 
That is, the space is regarded as a single sphere, a 
camera is installed at the center of the sphere, an image 
of half of the sphere is picked up by the fisheye lens, the 
camera is turned at an angle of 180° from that position, 
an image of the other half of the sphere in the opposite 
direction is picked up, and the two images are combined 
together to obtain an image of ail the. directions of the 
field of view in the space of 360°, that is, the sphere. 
This image is converted into a plane image. 
[0004] As the monitoring system comprising a fish- 
eye lens of the prior art, there is a system disclosed by 
Japanese Patent Application Laid-open No. Hei6- 
501585 (to be referred to as "prior art" hereinafter), for 
example. Although this prior art makes it possible to pick 
up an image of all the directions of the f ield of view, the 
lens used in the prior art is a fisheye lens having a rela- 
tionship of h b f • 6 (wherein h is the height of an 
image of a subject at a certain point obtained by the 
fisheye lens, f is the focal distance of the fisheye lens 
and 0 is a field angle). This is obvious from the fact that 
Nikon's 8-mm f/2.8 lens is used as the fisheye lens in 
the above Japanese Patent Application Laid-open No. 
Hei6-501585. Conventional fisheye lenses generally 
have a relationship of h = f • 0 and Nikon's 8-mm f/2.8 
fisheye lens has the above relationship of h = f • 0 . 
[0005] The method of picking up an image by a fish- 
eye lens having this relationship of h = f • e and con- 
verting the image into a plane image is called 
"equidistant projection". Since an image picked up by a 
fisheye lens having the above characteristics has a 
small volume of image data on its peripheral portion 
(field angle of around 90° with respect to the optical axis 
of the fisheye lens), when the image is converted into a 
plane image, there are many missing portions of image 
data on the peripheral portion of the image and the 
missing portions must be interpolated. In addition, the 



image picked up by the fisheye lens having the above 
characteristics involves such a problem that the periph- 
eral portion of the image is distorted. 
[0006] An object of the present invention is to pro- 

5 vide an image pick-up device comprising a fisheye lens, 
an image display device and an information recording 
medium, which minimize missing portions of image data 
by extracting a large volume of image data at a field 
angle of around 90° with respect to the optical axis of 

io the fisheye lens to reduce interpolating of the missing 
portions and can obtain a natural plane image when 
images of all the directions of the field of view around 
the optical axis are picked up at a field angle of at least 
90° with respect to the optical axis and are converted 

15 into plane images. 

[0007] Various other objects, advantages and fea- 
tures of the present invention will become readily appar- 
ent to those of ordinary skill in the art, and the novel 
features will be particularly pointed out in the appended' 

20 claims. 

[0008] To attain the above object, according to a 
first aspect of the present invention, there is provided an 
image pick-up device comprising a fisheye lens for pick- 
ing up an image of all the directions of the field of view 

25 around the optical axis of the fisheye lens at a field 
angle of at least 90° in each direction with respect to the 
optical axis, wherein the fisheye lens has a relationship 
of h = nf • tan(6/m) (wherein h is the height of an 
image of a subject at a certain point obtained by the 

30 fisheye lens, f is the focal distance of the fisheye lens, 
1 .6^ m < 3, m-0.4 < n < m + 0.4 , and 8 is a field angle). 
[0009] According to a second aspect of the present 
invention, the fisheye lens is constructed by a master 
lens provided on an existing image pick-up device and 

35 by an attachment lens to be attached to the master lens. 
[0010] Further, according to a third aspect of the 
present invention, there is provided an image display 
device comprising an image data processing unit for 
converting an image obtained by the image pick-up 

40 device of the first or second aspect of the present inven- 
tion into a plane image and a display unit for displaying 
the converted plane image. 

[0011] According to a fourth aspect of the present 
invention, there is provided an information recording 

45 medium that records a program having at least the step 
of converting an image obtained by a fisheye lens hav- 
ing a relationship of h = nf • tan(e/m) (wherein h is the 
height of an image of a subject at a certain point, f is the 
focal distance of the fisheye lens, 0 is a field angle, 

so 1 .6^m^3. and m-0.4^n^m+0.4 ) into a plane image, the 
step of displaying a predetermined portion of the con- 
verted plane image on a display unit and the step of 
changing continuously the predetermined portion with 
instruction means. 

55 [001 2] .One of the fisheye lens used in the present 
invention has the relationship of h = 2f - tan(e/2) . 
Compared with an ordinary fisheye lens having a rela- 
tionship of h = f • e , an image at a peripheral portion 
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(field angle of around 90° with respect to the optical axis 
of the f isheye lens) is enlarged and missing portions of 
image data on the peripheral portion can be minimized 
with the fisheye lens in accordance with the present 
invention. With this, when a picked-up image is to be 5 
converted into a plane image, the interpolating of image 
data can be reduced, thereby making it possible to 
obtain a more natural plane image. 
[0013] The fisheye lens according to the present 
invention may be constructed by attaching an attach- 10 
ment lens to a master lens provided on an existing cam- 
era so that the fisheye lens can be attached to almost all 
the existing cameras. In addition, only the attachment 
lens is newly produced, thereby making it possible to 
reduce costs. is 
[001 4] Further, the image display device for display- 
ing a plane image converted from an image picked up 
by the image pick-up device having the above fisheye 
lens on a display unit makes the displayed image easy 
to be seen, thereby improving the value of the device. 20 
When the information recording medium recording the 
above steps is read by a computer and the program is 
executed, a more natural plane image can be displayed 
on the display unit and the displayed portion can be 
freely shifted within the range of the image picked up by 25 
the image pick-up device. 

[0015] Embodiments of the invention will now be 
described, by way of example, with reference to the 
drawings, wherein like reference numerals denote like 
elements and parts, in which: 30 

Fig. 1 is a schematic structural diagram of an image 
processing system using an image pick-up device 
comprising a fisheye lens according to the present 
invention; 35 

Figs. 2 are a structural diagram of the fisheye lens 
shown in Fig. 1 and a corresponding diagram sche- 
matically showing lens intervals (lens intervals and 
lens thicknesses); *o 

Fig. 3(A) is a diagram showing the relationship 
between field angle 6 and image height h with 
respect to fisheye lenses having relationships of 
h o f • 6 , h = 2f • sin (6/2) , h = f • sine , 45 
h = f • tan 0 , h=3f • tan (0/3) , h=2f • (tan e/1 .6) 
and one of the fisheye lenses of the present inven- 
tion having a relationship of h = 2f • tan(0/2) ; 

Fig. 3(B) is a diagram showing the relationship 50 
between field angle e and image height h with 
respect to fisheye lenses having relationships of 
h=2f • tan(0/1 .6) . h= 2.4f • tan(8/2) , 

h = 3.4f • tan(0/3) . h = 1 .2f • tan(8/1 .6) . 

h = 1.6f • tan(0/2) and h = f • 6 . 55 

Figs. 4(A) to 4(D) are views illustrating, in concen- 
tric circles each centering around the optical axis of 



each fisheye lens shown in Fig. 3, changes of 
image heights when the field angle is changed in 
10° with respect to the optical axis of each fisheye 
lens; 

Fig. 5 is a diagram for explaining a method of polar- 
coordinate converting a hemispherical image 
obtained by a fisheye lens; 

Fig. 6 is a diagram for explaining a method of 
obtaining the position of an image formation point 
on the surface of CCD image pick-up elements in 
the polar coordinate conversion of Fig. 5; 

Fig. 7 is a flow chart for explaining the steps of 
processing an image using the image processing 
system of Fig. 1 ; 

Figs. 8(A) and 8(B) are diagrams showing another 
application example of the image pick-up device of 
the present invention, wherein Fig. 8(A) is a sche- 
matic diagram showing the side thereof and Fig. 
8(B) is a diagram when seen from a direction indi- 
cated by an arrow B in Fig. 8(A); 

Figs. 9(A) and 9(B) are diagrams showing still 
another application example of the image pick-up 
device of the present invention, wherein Fig. 9(A) is 
a schematic diagram showing the side thereof and 
Fig. 9(B) is a diagram when seen from a direction 
indicated by an arrow B in Fig. 9(A); and 

Fig. 10 is a diagram showing another example of an 
image processing system utilizing the image pick- 
up device comprising a fisheye lens of the present 
invention. 

[001 6] Preferred embodiments of the present inven- 
tion will be described hereinafter with reference to Figs. 
1 to 10. 

[0017] Fig. 1 schematically shows an image 
processing system utilizing an image pick-up device 
comprising a fisheye lens of the present invention. This 
image processing system comprises a camera (such as 
a video camera) 2 that is an image pick-up device 
equipped with a fisheye lens 1 , an image data process- 
ing unit 3 for processing image data from the camera 2, 
and a monitor unit 4 for displaying an image processed 
by the image data processing unit 3. The image data 
processing unit 3 has a CPU, memory means and the 
like and performs various processing using image data 
output from the camera 2. In the case of the present 
invention, the image data processing unit 3 also con- 
verts an image picked up by the fisheye lens 1 into a 
plane image. 

[0018] As shown in Fig. 2, the fisheye lens 1 used in 
this embodiment roughly consists of a lens unit (called 
master lens unit) 10 provided on the camera 2 and a 
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lens unit (called attachment lens unit) 20 that can be 
attached to and detached from the master lens unit 10. 
The f isheye lens 1 of the present invention functions as 
a fisheye lens when the attachment lens unit 20 is 
attached to the master lens unit 10. 
[0019] The attachment lens unit 20 consists of a 
first lens 21, a second lens 22, a third lens 23, a fourth 
lens 24 and a plate 25. The master lens unit 1 0 consists 
of a fifth lens 11, a sixth lens 12, a seventh lens 13, an 
eighth lens 14, a ninth lens 15 and a diaphragm 26 inter- 
posed between the sixth lens 12 and the seventh lens 
13. 

[0020] The curvature R (diameter of the curved sur- 
face of the lens) of each lens and interval D (lens thick- 
ness or lens interval) in this embodiment are as follows. 
That is, beginning with the curvature R1 of the left 
curved surface of the first lens 21 on the leftmost side of 
Figs. 2, in turn, the curvatures R1 and R2 of the first 
lens 21 are 40.0 mm and 9.0 mm, the curvatures R3 
and R4 of the second lens 22 are -26.0 mm and 80.0 
mm, the curvatures R5 and R6 of the third lens 23 are - 
36.0 mm and -20.0 mm, and the curvatures R7 and R8 
of the fourth lens 24 are -81.0 mm and -27.0 mm, 
respectively. 

[0021] Further, the curvatures R9 and R10 of the 
fifth lens 1 1 are 14.0 mm and 68.0 mm, the curvatures 
R11 and R12 of the sixth lens 12 are 9.0 mm and 3.0 
mm. the curvatures R13 and R14 of the seventh lens 13 
are 0.0 mm and -8.0 mm, the curvatures R1 5 and R16 
of the eighth lens 1 4 are 1 0.0 mm and -6.0 mm, and the 
curvatures R17 and R1 8 of the ninth lens 15 are 11.0 
mm and -9.0 mm, respectively. 
[0022] Meanwhile, the thickness D1 of the first lens 
21 on the leftmost side of Figs. 2 is 1 .2 mm, the interval 
D2 between the first lens 21 and the second lens 22 is 
10.00 mm, and the thickness D3 of the second lens 22 
is 1.2 mm. The interval D4 between the second lens 22 
and the third lens 23 is 14.0 mm, the thickness D5 of the 
third lens 23 is 2.0 mm, the interval D6 between the third 
lens 23 and the fourth lens 24 is 3.0 mm, and the thick- 
ness D7 of the fourth lens 24 is 5.0 mm. 
[0023] Further, the interval D8 between the fourth 
lens 24 and the fifth lens 1 1 is 7.0 mm; the thickness D9 
of the fifth lens 1 1 is 2.0 mm, the interval D10 between 
the fifth lens 1 1 and the sixth lens 12 is 0.3 mm, and the 
thickness D1 1 of the sixth lens 12 is 0.8 mm. The sev- 
enth lens 13, the eighth lens 14 and the ninth lens 15 
can be moved in the direction of the optical axis to 
change magnification, and the intervals between adja- 
cent lenses to be described hereinafter are maximum 
values thereof. The interval D12 between the dia- 
phragm 26 and the seventh lens 13 is 4.0 mm, the thick- 
ness D1 3 of the seventh lens 13 is 1.0 mm, the interval 
D14 of the seventh lens 13 and the eighth lens 14 is 1.0 
mm, and the thickness D15 of the eighth lens 14 is 4.0 
mm. 

[0024] The interval D1 6 between the eighth lens 14 
and the ninth lens 15 is 2.0 mm, and the thickness D17 



of the ninth lens is 4.0 mm. Parallel plates 1 6 and 1 7 are 
arranged on the right side in Figs. 2 of the ninth lens 1 5. 
[0025] In this arrangement, light incident upon the 
first lens 21 passes through the first to fourth lenses 21 

s to 24, further through the fifth to ninth lenses 1 1 to 15 
and is input into CCD image pick-up elements 30 in the 
camera 2. In this attachment lens unit 20, parallel rays 
input into the first lens 21 are output from the fourth lens 
24 as parallel rays. Therefore, this attachment lens unit 

10 20 can be attached to almost all the cameras. The width 
of the parallel pencil of rays output from the fourth lens 
24 of the attachment lens unit 20 (shown by "w" in the 
figure) is set to V2 or less the effective diameter of the 
master lens 10 of the camera to which the attachment 

15 lens unit 20 is attached. In Fig. 2, the spherical surface 
40 at the front of the first lens 21 represents a virtual 
subject surface of the picked up image. 
[0026] As described above, the present invention is 
characterized in that a desired fisheye lens 1 is con- 

20 structed by the master lens unit 10 and the attachment 
lens unit 20 and has a relationship of h = 2f • tan(9/2) 
(wherein h is the height of an image of a subject at a 
certain position, f is the focal distance of the fisheye lens 
and 6 is a field angle). It is noted that although the pre- 

25 ferred embodiment of the present invention has the 
above-indicated relationship, the present invention also 
embodies fisheye lenses having the relationship of 
h = n • f • tan(e/m) , where m has the value of 1 .6< m 
< 3, and n has the value of m - 0 4 < n ^ m + 0.4 . Also, 

30 the present invention contemplates such a relationship 
when m equals n. Also, the relationship 
h = 1 .2 f • tan(e/m) , m > 1 .6 also is embodied by the 
present invention. However, for purposes of discussion 
herein, m and n both equal to 2. 

35 [0027] The fisheye lens that has been generally 
used in the prior art has a relationship of h = f • e as 
described above. These functions are used to map a 
spherical image as a polar-coordinate converted image. 
The relationship, other than that, may be 

40 h = 2f • sin(0/2) , h = 2f • sin 6 or h = f • tan 9 . 

[0028] Fig. 3(A) is a diagram showing relationships 
between field angle 6 and image height h when fisheye 
lenses having relationships of h = 2f • tan(0/2) , 
h = f • 0 , h m f • sin(0/2) , h = f • sin 6 

45 and h = f ♦ tan 6 , etc. are used. Here, e = 90° shows a 
field angle with respect to the optical axis (the field 
angle of the optical axis is 0°). 

[0029] In Fig: 3(A), a curve C1 shows the relation- 
ship between field angle e and image height h when the 

so fisheye lens of the present invention having the relation- 
ship of h = 2f • tan(e/2) is used, and a curve C2 shows 
the relationship between field angle 0 and image height 
h when a fisheye lens having the relationship of 
h = f • 0 is used. A curve C3 shows the relationship 

55 between field angle 9 and image height h when a fish- 
eye lens having the relationship of h = 2f • sin(9/2) is 
used, a curve C4 shows the relationship between field 
angle 0 and image height h when a fisheye lens having 
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the relationship of h = 2f ? sine is used, and a curve C5 
shows the relationship between field angle e and image 
height h when a fisheye lens having the relationship of 
h = f • tane is used. A curve C1 ' shows the relationship 
of h a 3f • tan (e/3) and a curve C" shows the relation- 
ship of h = 2f • tan (0/1 .6). 

[0030] Fig. 3(B), is a diagram showing the relation- 
ship between field angle e and image height when fish- 
eye lenses having relationships of h = 2f • tan (0/1.6) , 
h = 2.4f • tan(0/2) , h = 3.4f • tan (0/3) , 

h = 1 .2f • tan (0/1 .6) , h = 1 .6f • tan (0/2) and 
h = f • 0 .All of these fisheye lenses, except h = f • 0, 
are embodied by the present invention. 
[0031] As is evident from Fig. 3, an increase in 
image height h at a field angle 0 of about 90° is largest 
when the fisheye lens having the relationship of 
h =f • tanO is used and is second largest when the fish- 
eye lens having the relationship of h = 2f • tan(0/2) is 
used. Changes in image height h with respect to 
changes in field angle 0 become linear when the fisheye 
lens having the relationship of h = f • 0 is used and fur- 
ther an increase in image height h tends to be smaller 
as the field angle becomes closer to 90° when the fish- 
eye lenses having the relationships of h = 2f • sin(0/2) 
and h = f • sine are used. 

[0032] An increase in image height becomes larger 
toward the peripheral portion (field angle of 90°) when 
the fisheye lens having the relationship of h = f • tanO is 
used and more image data can be obtained. However, 
at a field angle 0 of 90°, tanO becomes infinite. Since 
the fisheye lens is required to obtain an image of all the 
directions of the field of view around the optical axis at a 
field angle of at least 90° with respect to the optical axis, 
it can be said that the fisheye lens having the relation- 
ship of h = f • tan© is not suitable. 
[0033] Therefore, the fisheye lenses having rela- 
tionships of h = 2f • tan(0/2) , h = f • 0 , h = 2f • sin(0/2) 
and h = f • sinO may be used. Figs. 4(A) to 4(D) show, 
in concentric circles each centering around the optical 
axis of each fisheye lens, image heights h when the field 
angle 0 is changed in 1 0° with respect to the optical axis 
of each fisheye lens. Fig. 4(A) shows the image height 
in the fisheye lens having the relationship of 
h = 2f • tan(0/2) , Fig. 4(B) shows the image height in 
the fisheye lens having the relationship of h = f • 0 , Fig. 
4(C) shows the image height in the fisheye lens having 
the relationship of h - 2f • sin(0/2) , and Fig. 4(D) 
shows the image height in the fisheye lens having the 
relationship of h = f • sine . In Figs. 4(A) to 4(D), h Q rep- 
resents the height of an image M 0 near the optical axis 
of each fisheye lens and h e represents the height of an 
image M e at afield angle of around 90°. 
[0034] As is understood from Figs. 4(A) to 4(D), 
image height at a field angle of around 90° when the 
fisheye lens having the relationship of h = 2f • sin(0/2) 
or the fisheye lens having the relationship of h = f • sinO 
is used is smaller than image height near the optical 
axis and only a small volume of image data can be 



obtained. The image height h e of an image M e at a 
peripheral portion of the fisheye lens that has been gen- 
erally used and has the relationship of h = f • 0 is the 
same as the image height h 0 of an image M Q near the 

5 optical axis and the image is distorted. 

[0035] From these facts, it can be said that the fish- 
eye lenses having relationships of h = 2f • sin(0/2) and 
h = f • sine are not preferred in view that how large vol- 
ume of data can be obtained at a field angle of 90° or 

10 therearound. Even with the fisheye lens that has been 
generally used and has the relationship of h = f • o is 
not satisfactory. 

[0036] In contrast to that, the image height h e of the 
image M e at a peripheral portion of the fisheye lens 1 

15 having the relationship of h = 2f • tan(6/2) in accord- 
ance with the present invention is enlarged and larger 
than the image height h Q of the image M Q near the opti- 
cal axis, a larger volume of image data can be obtained 
in comparison with the conventional fisheye lens, and 

20 the obtained image is not distorted. 

[0037] When a single spherical image obtained by 
combining two hemispherical images of all the direc- 
tions of the field of view around the optical axis of the 
fisheye lens 1, which are picked up at a field angle of 

25 90° with respect to the optical axis is converted into a 
plane image by the image data processing unit 3, it is 
necessary to interpolate missing image data on the 
peripheral portion (field angle of around 90° with 
respect to the optical axis) of the image. According to 

30 the present invention, since an image at the peripheral 
portion is enlarged and a large volume of data on the 
peripheral portion can be extracted, the volume of 
image data to be interpolated can be greatly reduced, 
when compared with the conventional system. 

35 [0038] An image of all the directions of the field of 
view around the optical axis is picked up at a field angle 
of at least 90° with respect to the optical axis and is 
polar-coordinate converted into a plane image in the fol- 
lowing manner. 

40 [0039] An X, Y and Z coordinate system as shown 
in Fig. 5 is imagined in subject space. At this point, the 
optical axis of the fisheye lens 1 is made 2 axis. The 
coordinates of a certain point p are represented as (X1 , 
Y1 , Z1) and the elevation angle of the point p from the 

45 origin O of the coordinates with respect to the XZ plane 
is represented by 0. The elevation angle of the point p 
from the position of Z1 on the Z axis with respect to the 
XZ plane is represented by <(>. 

[0040] When an x and y coordinate system having 
so the optical axis (Z axis) as an origin o is imagined on the 
surface of CCD image pick-up elements 30 as shown in 
Fig. 6 and the focal distance of the fisheye lens 1 is rep- 
resented by f, the image formation point (p') for the point 
p is located as shown in Fig. 6. In Fig. 6, iz is added to <t> 
55 because an image formed at the point p' is inverted ver- 
tically and horizontally wrth respect to the image of the 
subject surface (point p). The optical axis in Fig. 6 is 
present in a direction perpendicular to the paper from 
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the origin o of the x and y coordinates. 
[0041] The position of the point p' is expressed as 
polar coordinates with a length (h) between the origin o 
and the point p' and an angle <|> + n formed by op' and 
the x axis. When the polar coordinates are expressed 5 
on the x and y rectangular coordinates, the position (x1 , , 
y1) on the x and y rectangular coordinates are 
expressed as follows. 

x1 = h • cos((|> + n) (1) 10 

y1 ■ h • sin(<|> + n) (2) 

In addition, the image height h of the point p* is repre- 
sented by h = 2f • tan(6/2) , hence, when 15 
h = 2f • tan(6/2) is substituted into the above expres- 
sions (1) and (2), the coordinates (x1 , y1) of the image 
formation point p' on the surface of the CCD image pick- 
up elements 30 are as follows. 

20 

xl = 2f • tan(0/2) • cos(<|>+70 (3) 
y1 = 2f • tan(0/2) • sin(^+7t) (4) 
As a result, they are expressed as follows. 25 
x1 = -2f • tan(e/2) • cos* (5) 
y1 = -2f • tan(G/2) • sin* (6) 

30 

In the above expressions, 6 and <|> are defined as fol- 
lows. 

6=tan" 1 (X1 2 +Y1 2 /Z1) 

35 

(t>=tan 1 (Y1/X1) 

[0042] Thus, the position of the point p' on the sur- 
face of CCD image pick-up elements 30 can be 
obtained for the point p on the surface of the subject. 40 
[0043] Thereafter, a .description is subsequently 
given, with reference to Fig. 7, of steps required when 
the sphere (all directions) is photographed by the cam- 
era 2 comprising the fisheye lens 1 and an image 
thereof is displayed on the monitor unit 4 that is the dis- 45 
play unit. 

[0044] Briefly the steps S1 to S7 are as follows: 

51 photographing a hemisphere with a camera 2 
equipped with a fisheye lens so 

52 photographing the other hemisphere with the 
camera 2 equipped with a fisheye lens 

53 combining two images that are polar-coordinate 55 
displayed and converting the combined image into 

a plane image on the surface of the CCD image 
pick-up elements 30 as a polar-coordinate con- 



verted image. 

[0045] The above two images are then combined 
together and the combined image is converted into a 
plane image by the image data processing unit 3 (step 
S3). At this point, an area corresponding to a connec- 
tion portion between these hemispheres must be cor- 
rected. Since the polar-coordinate converted image 
obtained by the fisheye lens 1 has a large volume of 
information on a peripheral portion, the processing of 
combining these images is easy. Thereafter, a predeter- 
mined portion of the thus obtained plane image is 
extracted and displayed on the monitor unit 4 (step S4). 
[0046] A user shifts the screen with instruction 
means such as a mouse when the user likes to change 
the displayed predetermined portion. This shifting can 
be made continuously in any direction of 360° around 
the portion displayed on the monitor unit 4 (step S5). 
[0047] The above steps are for picking up an image 
of a sphere in all the directions of 360°. When only a sin- 
gle hemisphere is photographed, the same steps are 
taken. However, step S2 is unnecessary and the 
processing of combining two images in step S3 is also 
unnecessary. 

[0048] In the present invention, since the volume of 
information on the peripheral portion of an image 
obtained by the fisheye lens 1 is large, that is, an image 
at the peripheral portion is enlarged, it is convenient 
when the present invention is used for the examination 
of a product. For example, when the inner surface 52 of 
a cylindrical body 51 is photographed at the condition 
that the optical axis of the fisheye lens 1 of the present 
invention is aligned with the central axis of the cylindri- 
cal body 51 as shown in Figs. 8(A) and 8(B), a periph- 
eral portion of an image can be extracted as an image 
having a larger volume of information than a central por- 
tion in the present invention. Therefore, it is easy to find 
that a scratch has been generated in the inner surface 
52. Consequently, this can be used for the examination 
of a pipe-like body such as a water pipe or gas pipe and 
further for the monitoring of a crack that has been gen- 
erated in the wall surface of a tunnel or the like. 
[0049] It can also be used for the examination of the 
connection condition of a small part such as an IC. That 
is, when a part 71 is fixed to a substrate 74 by a solder 
73 on its both sides 72 and 72 as shown in Figs. 9(A) 
and 9(B), the soldering state of the part must be 
checked from its side direction in the prior art. However, 
when another part 75 is existent in a side direction, the 
solder 73 on the part 75 side cannot be seen through a 
camera, thereby making automatic examination difficult. 
On the other hand, even when the fisheye lens 1 is 
installed right over the part 71 as shown in Fig. 9(A), the 
side direction of the part 71 can be sufficiently photo- 
graphed by the camera 2 comprising the fisheye lens 1 
of the present invention, thereby enabling automatic 
examination with the camera 2. - 
[0050] Although each foregoing embodiment is an 



BNSDOCID: <EP : 1 00491 5A1 i > 



-6 



11 



EP 1 004 915 A1 



12 



example of a preferred embodiment of the invention, it is 
to be understood that the invention is not limited thereto 
and that various changes and modifications may be 
made in the invention. For example, the fisheye lens 
may be constructed by the attachment lens unit 20 5 
alone without the master lens 10, or contrariwise may 
be constructed by an integrated unit of the master lens 
unit 10 and the attachment lens unit 20. Also, the con- 
struction and numerical values of the fisheye lens 1 
shown in the above embodiment are just examples and 70 
a fisheye lens having other construction and numerical 
values may be used. 

[0051] Further, as the system comprising the fish- 
eye lens 1 of the present invention, an image process- 
ing system 81 shown in Fig. 10 may be used. This 15 
image processing system 81 is mainly constructed with 
a camera 2 equipped with a fisheye lens 1 and an image 
data processing unit/monitor 5 connected to the camera 
2 by a cable. The image data processing unit/monitor 5 
is a personal computer equipped with a monitor, and a 20 
key board 5a and a mouse 5b are connected to the 
computer as instruction means. 
[0052] Also, this image data processing unit/moni- 
tor 5 has a hard disk (not shown) in that the contents of 
an information recording medium (floppy disk) 6 record- 25 
ing a program for executing the steps S3, S4 and S5 
shown in Fig. 7 are to be installed. By installing this pro- 
gram in the image data processing unit/monitor 5, the 
image data processing unit/monitor 5 carries out the 
same function as the image data processing unit 3 30 
which has been described in the foregoing. 
[0053] Image pick-up data may be transferred from 
the camera 2 to the image data processing unit/monitor 
5 by a memory card such as a flash card or wireless 
communication such as infrared communication, 35 
besides a cable. Further, the program may be installed 
not from the f bppy disk 6 but other recording medium 
such as a CD-ROM, or transferred from other storage 
unit over a network When the program is transferred 
over a network, the storage unit of a transmitter or the 40 
hard disk (storage unit) of the image data processing 
unit/monitor 5 serves as the information recording 
medium of the present invention. 
[0054] As having been described above, in the 
image pick-up device comprising the fisheye lens 45 
according to the first aspect of the present invention, the 
fisheye lens has the relationship of h = nf • tan(0/m) 
(wherein h is an image height, f is a focal distance, and 
e is a field angle). With this, compared with the ordinary 
fisheye lens having the relationship of h = f • 0 (wherein so 
h is an image height, f is a focal distance, and 0 is a field 
angle), an image at a peripheral portion (field angle of 
around 90° with respect to the optical axis of the fisheye 
lens) is enlarged and the volume of information is large, 
thereby making it possible to minimize the missing por- 55 
tions of image data on the peripheral portion. When the 
picked-up image is to be converted into a plane image, 
interpolating of image data can be thereby minimized 



and a more natural plane image can be obtained. 
[0055] Also, according to the second aspect of the 
present invention, by attaching the attachment lens dif- 
ferent from the master lens provided on the existing 
image pick-up device (camera) to the master lens, the 
fisheye lens is constructed by this master lens and the 
attachment lens. Therefore, the fisheye lens can be 
attached to almost all the existing image pick-up 
devices (cameras) and further, only the attachment lens 
is newly produced, thereby reducing costs. 
[0056] Further, according to the third aspect of the 
present invention, the image display device can convert 
a spherical image into a plane image with ease and can 
display a more natural plane image. In addition, accord- 
ing to the fourth aspect of the present invention, when 
the program recorded in the information recording 
medium is read and executed by a computer, a more 
natural plane image can be formed. 

Claims 

1 . An image pick-up device, comprising: 

a fisheye lens for picking up an image of all 
directions of a field of view around an optical 
axis of said fisheye lens at a field angle of at 
least 90° in each direction with respect to the 
optical axis 

the fisheye lens having a relationship of 
h = n • f • tan(0/m) , h being a height of an 
image of a subject at a predetermined point 
obtained by the f isheye lens, f being a focal dis- 
tance of the fisheye lens, e being a field angle, 
m having a value of 1 .6 <m^ 3 t and n having a 
value of m- 0.4< n < m+0.4. 

2. The image pick-up device of Claim 1 , wherein the 
fisheye lens is constructed by a master lens pro- 
vided on an existing image pick-up device and an 
attachment lens to be attached to the master lens. 

3. An image display device, comprising: 

an image data processing unit for converting 

an image obtained by the image pick-up device 

of Claim 1 into a plane image; and 

a display unit for displaying the converted plane 

image. 

4. The image display device of Claim 7, further com- 
prising a fisheye lens for picking up an image of all 
directions of a field of view around an optical axis of 
said fisheye lens at a field angle of at least 90° in 
each direction with respect to the optical axis, the 
fisheye lens having a relationship of 
h = n • f • tan(8/m) , h being a height of an image 
of a subject at a predetermined point obtained by 
the fisheye lens, f being a focal distance of the f ish- 
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eye lens, e being a field angle, m having a value of 
1.6 £m ^3, and n having a value of m-0.4 
^n<m+0.4 

wherein said image data processing unit 
receives the image picked up by said fisheye lens. 5 

5. A method of converting into a plane image an 
image generated by a fisheye lens comprising the 
steps of: 

10 

converting into a plane image an image 
obtained by a fisheye lens having a relationship 
of h = n • f • tan(e/m) , h being a height of an 
image of a subject at a predetermined point 
obtained by the fisheye lens, f being a focal dis- is 
tance of the fisheye lens, 6 being a field angle, 
m having a value of 1 .6 ^m ^ 3, and n having a 
value of m-0.4 ^n ^ m + 0.4 ; 
displaying a predetermined portion of the con- 
verted plane image on a display unit; and 20 
changing continuously the predetermined por- 
tion with instruction means. 

6. The image pick-up device of claim 1 , image display 
device of claim 3 or method of claim 5 wherein m 25 
and n are substantially equal. 

7. The image pick-up device of claim 1 , image display 
device of claim 3 or method of claim 5 wherein m 
and n both substantially equal 2. 30 

8. The image pick-up device of claim 1 , image display 
device of claim 3 or method of claim 5 wherein m 
and n both equal 2 such that the fisheye lens has a 
relationship of h = 2f • tan (6/2) . 35 

9. A recording medium having recorded thereon a 
program for implementing the method of any of 
claims 5 to 8. 

40 

10. An optical system for attachment to an optical 
device to form an image pick-up device as claimed 
in any of claims 1 to 2 or 6 to 8. 

1 1. An image pick-up device, comprising: 45 

a fisheye lens for picking up an image of all 
directions of a field of view around an optical 
axis of said fisheye lens at a field angle of at 
least 90° in each direction with respect to the so 
optical axis 

the fisheye lens having a relationship of 
h = 1 .2f • tan(0/m) , h being a height of an 
image of a subject at a predetermined point 
obtained by the fisheye lens, f being a focal dis- 55 
tance of the fisheye lens, 0 being a field angle, 
and m having a value of m< 1 .6. 
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